Method for classifying a digital image

ABSTRACT

A method for associating with a digital image a class of a plurality of predefined classes having respective models, the method including the phases of dividing the digital image pixel by pixel into one or more regions belonging to a set of predefined regions that differ from each other on account of their type of content, the division being effected by establishing whether or not a pixel of the image belongs to a respective region on the basis of an operation of analyzing the parameters this pixel, the analysis operation being carried out by verifying that the parameters satisfy predefined conditions and/or logico-mathematical relationships of belonging to the respective region, acquiring from the digital image divided into regions information regarding the regions that are present in it, comparing this information with at least one model characterizing a respective class of said plurality, and associating with the digital image a class on the basis of the comparison phase.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to the processing of digital images and,more particularly, to a method for classifying a digital image.

2. Description of the Related Art

Digital images are currently used in numerous applications, for examplein new generation acquisition devices such as photo cameras and digitalstill cameras (DSC). Moreover, digital images are being ever moreextensively employed in applications in the field of artificial visionand in applications in the field of medical diagnostics.

In these and other contexts image classification plays a fundamentalrole as integrating part of the processing procedures, because it makesit possible not only to select the most appropriate algorithms on thebasis of the type of image that has to be processed, but also to renderthe processing procedures independent of interaction with the user.

In the field of artificial vision, for example, some techniques forconverting two-dimensional images into three-dimensional images aresensitively bound up with the type of image to be converted.

In some of these conversion techniques, in particular, there istypically generated a depth map relating to the 2D image to beconverted, and on the basis of this map there is subsequently created astereoscopic pair necessary for the reconstruction of the 3D outputimage. In the generation of this depth map the classification of theimages acts as a “guide” element within the system, because it makes itpossible to select and therefore utilize the most appropriate techniqueson the basis of the image that is to be processed.

Image classification also plays a very important part in the processingprocedures performed within the digital image acquisition devices. Inthis context the classification is utilized, for example, to optimizethe processing as regards improvement of the quality of an acquireddigital image or to optimize the compression-encoding operations.

Though widely used, the known classification techniques are associatedwith limits both as regards the precision of the results and from thepoint of view of computational complexity.

BRIEF SUMMARY OF THE INVENTION

The disclosed embodiments of the present invention therefore provide arobust classification method that will be capable of providing betterperformance as compared to the methods of the known techniques.

In accordance with one embodiment of the invention, a method forassociating with a digital image a class of a plurality of predefinedclasses having respective models is provided. The method includes thephases of dividing the digital image pixel by pixel into one or moreregions belonging to a set of predefined regions that differ from eachother on account of their type of content, the division being effectedby establishing whether or not a pixel of the image belongs to arespective region on the basis of an operation of analyzing theparameters this pixel, the analysis operation being carried out byverifying that the parameters satisfy predefined conditions and/orlogico-mathematical relationships of belonging to the respective region;acquiring from the digital image divided into regions informationregarding the regions that are present in it; and comparing thisinformation with at least one model characterizing a respective class ofthe plurality of classes, and associating with the digital image a classon the basis of the comparison phase.

In accordance with another embodiment of the invention, a method forclassifying a digital image in a class from a plurality of classes, eachclass having a respective model associated therewith, is provided. Themethod includes analyzing a digital image pixel by pixel to determineparameters of each pixel by verifying that each parameter satisfies atleast one of pre-established conditions, logico-mathematicalrelationships, or both pre-established conditions andlogico-mathematical relationships that are associated with respectivepredefined regions, and dividing the pixels into one or more regionsbased on the analysis; acquiring from the digital image informationrelating to the regions found in the digital image; comparing theacquired information with at least one model; and associating with thedigital image a class on the basis of the comparison.

In accordance with yet another embodiment of the invention, the methodincludes the phases of receiving an input image in CFA format andperforming a chromatic reconstruction on the input image to provide anoutput image; segmenting the output image by dividing the output imageinto one or more areas that are chromatically substantially homogeneousand comprising a reduced range of colors as compared with the outputimage to form an output digital image; dividing the digital image pixelby pixel into one or more regions by analyzing the image pixel by pixelto determine parameters of each pixel and verifying that each parametersatisfies at least one of pre-established conditions,logico-mathematical relationships, or both pre-established conditionsand logico-mathematical relationships that are associated withrespective predefined regions; acquiring from the digital imageinformation relating to the regions found in the digital image;comparing the acquired information with at least one model; andassociating with the digital image a class on the basis of thecomparison.

In accordance with still yet a further embodiment of the invention, amethod for classifying a digital image in a class from a plurality ofclasses is provided, each class having a respective model associatedtherewith. The method includes the steps of receiving an input image inCFA format and performing a chromatic reconstruction on the input imageto provide an output image; segmenting the output image by dividing theoutput image into one or more areas that are chromatically substantiallyhomogeneous and comprising a reduced range of colors as compared withthe output image to form an output digital image; dividing the digitalimage pixel by pixel into one or more regions belonging to a set ofregions that are predefined and different from each other on the basisof their type of content by: application of a median filter to thedigital image to obtain a filtered digital image; extraction of thedigital values associated with RGB and HSI components of the filtereddigital image; assignment to each pixel of the filtered digital image ofa respective gray level for obtaining a provisional gray-level imagedivided into regions; and application of a median filter to theprovisional gray-level image for obtaining a final image divided intogray-level regions; acquiring from the digital image informationrelating to the regions found in the digital image; comparing theacquired information with at least one model; and associating with thedigital image a class on the basis of the comparison.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Further characteristics and advantages of the invention will be morereadily understood from the following detailed description of itspreferred embodiments, which is given by way of example and is not beunderstood as limitative in any way, of which:

FIG. 1 schematically shows a block diagram relating to a classificationmethod in accordance with the present invention,

FIG. 2 shows the pattern in which the filter elements are arranged in asensor with an optical Bayer filter,

FIGS. 3A and 3B show an input image of the segmentation block Img_Segand a corresponding segmented image produced as output from that block,

FIG. 4 shows the simplified flow chart of a preferred embodiment of theoperation of assigning a gray level I_(r)(x,y) to a pixel p(x,y) of thesegmented and filtered image F_Img_(s),

FIGS. 5A through 5C show an output image of the color reconstructionblock and the corresponding output images of, respectively, thesegmentation block and the block that divides the image into regions,

FIG. 6 shows the class decision block in greater detail by means of aflow chart, and

FIG. 7 shows the scanned columns in an image divided into regions in theinformation acquisition operation f₁ shown in the diagram of FIG. 6.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 schematically represents a block diagram of a classificationmethod Img_Class in accordance with a preferred embodiment of thepresent invention

As shown in FIG. 1, the classification method Img_Class preferablycomprises a color reconstruction block Col_Rec, a segmentation blockImg_Seg, a block Reg_Detect for dividing the image into regions, and aclass decision block class. Ideally, the method is implemented in acomputer system, such as a digital processing system for processing acomputer program configured to carry out the method of the invention.

The color reconstruction block Col_Rec is adapted to carry out aprocessing phase of chromatic reconstruction, by means of which,starting from a digital input image Img_(CFA) in CFA (Color FilterArray) format, i.e. including pixels that are each associated with arespective digital value (represented, for example, by 8 bits), oneobtains a complete digital output image I_(RGB), i.e. including pixelsthat are each associated with a respective triad of digital values.

The input digital image Img_(CFA) of the color reconstruction blockCol_Rec is preferably a digital image acquired with a sensor of the CCD(Charge Coupled Device) or CMOS (Complementary Metal OxideSemiconductor) type comprising an optical CFA filter, a Bayer matrixfilter for example.

As is well known to persons skilled in the art, only a singlephotosensitive cell is available in a sensor with a CFA filter foracquiring a pixel of the image. The sensor is covered by an opticalfilter comprising a matrix (a Bayer matrix, for example) of filterelements, each of which is associated with a photosensitive cell. Eachfilter element transmits to the photoelectric cell associated with itlight radiation corresponding to the wavelength of only red light, onlygreen light or only blue light (absorbing only a minimal part of it) andthus reveals only one component of each pixel.

The layout pattern BP of the filter elements in a Bayer filter isrepresented in FIG. 2, where R, G and B indicate, respectively, the red,green and blue filter elements.

The chromatic reconstruction phase is then used to recover at leastpartially the chromatic information present in the real scene.Preferably, the output image Img_(RGB) of the reconstruction block is adigital image in RGB (Red Green Blue) format, i.e. an image in whichthree digital values corresponding, respectively, to the chromaticcomponents R (Red), G (Green) and B (Blue) are associated with eachpixel.

In the chromatic reconstruction phase in order to obtain an output imageImg_(RGB) in RGB format from an input image Img_(CFA) in Bayer CFAformat there is carried out a color interpolation operation that isknown to persons skilled in the art and will not therefore be furtherdescribed.

In a particularly advantageous variant embodiment, the chromaticreconstruction phase comprises an operation of sub-sampling the inputimage Img_(CFA) in CFA format.

Preferably, in the sub-sampling operation the digital input imageImg_(CFA) is segmented into small adjacent and not overlapping blocks ofpixels, for example into blocks of size 2×2. Starting from each of thesesmall blocks, there is generated a respective colored pixel in theoutput image Img_(RGB) in RGB format, of which the R and B componentscorrespond to the digital values of the pixels on the secondary diagonalof the block and of which the G component is equal to the mean of thedigital values of the pixels on the principal diagonal of the block.

In this way, starting from each block of size 2×2 of the input imageImg_(CFA) in Bayer CFA format, the color reconstruction phase produces asingle pixel of the output image Img_(RGB) in RGB format.

The resolution of the output image is therefore reduced with respect tothe resolution of the input image. In this specific example it is equalto a quarter of the resolution of the input image.

Referring to the block pattern represented in FIG. 1, the segmentationblock Img_Seg that receives the input image Img_(RGB) is adapted toproduce a segmented output image Img_(s), i.e. divided into areas thatare substantially homogeneous from the point of view of chromaticappearance.

In greater detail, in the segmentation block Img_Seg there is carriedout a processing phase that makes it possible to identify thesignificant chromatic characteristics present in an image. Preferably,this processing phase is such as to operate in accordance with atechnique known by the name of “mean shift algorithm,” a simple andnon-parametric procedure for estimating the density gradients.Advantageously, this technique is found to be particularly efficientfrom the computational point of view. Nevertheless, other types ofsegmentation may also be utilized.

The mean shift algorithm technique is described, for example, in thedocument: “Robust Analysis of Feature Spaces: Color Image Segmentation”(D. Comaniciu, P. Meer, Department of Electrical and ComputerEngineering, Rutgers University, Piscataway, U.S.A.).

In accordance with this technique, the processing phase carried out inthe segmentation block Img_Seg preferably includes a preliminary phaseof transforming the digital values of the pixels of the image in RGBformat from the chromatic RGB space to the Luv space.

The passage from the RGB space to the Luv space calls for a non-lineartransformation.

As explained in the aforementioned article by Comaniciu and Meer, thistransformation operation makes it possible for the processing operationscarried out in the segmentation block to operate advantageously on asufficiently isotropic space of the components.

The processing phase carried out in the segmentation block Img_Seg alsoincludes a successive operation of grouping the pixels in an area on thebasis of their resemblance, that is to say, the homogeneity of thepixels transformed in this manner.

Lastly, a successive operation included in the processing phase is suchas to modify the pixels belonging to one and the same area in such amanner as to associate with them digital values that are substantiallyrepresentative of the predominant colors of the area, thereby obtaininga segmented image Img_(S) that, as compared with the image Img_(RGB)present at the input of the segmentation block Img_Seg, comprises areduced range of colors.

Referring to the aforementioned publication by Comaniciu and Meer, theprocessing phase performed in the segmentation block Img_Seg operates inaccordance with the mean shift algorithm technique having a lowsegmentation resolution (under-segmentation).

According to this technique, the pixels are grouped together in areas bymeans of homogeneity criteria having an ample margin of tolerance andthus obtaining a segmented image in which the pixels have associatedwith them only the most significant (i.e. predominant) colors present inthe input image Img_(RGB) of the segmentation block.

It should be borne in mind that the number of colors present in ageneric image depends on the quantity of detail present and is typicallycomprised between 35,000 and 40,000.

In a particularly preferred embodiment the segmentation block Img_Segproduces an output image Img_(S) in RGB format comprising at the mostabout ten colors and is therefore more homogeneous and compact than theinput image Img_(RGB).

In FIG. 3 Img_(RGB) is used to indicate an input image of thesegmentation block Img_Seg, while Img_(S) indicates a correspondingsegmented output image produced by means of an under-segmentationprocessing phase. Although the images in FIGS. 3A and 3B are representedin gray scale, one can nevertheless observe the effect of the colorreduction from the input image (in this example with 37,656 colors) tothe output image (in this example with 4 colors).

Advantageously, it has been found that the segmentation, andparticularly the “under-segmentation,” makes it possible to simplify thework of the subsequent block and, more particularly, makes it possibleto reduce the computational cost and at the same time to increase therobustness of the classification method in accordance with the presentinvention.

Coming back to the block diagram shown in FIG. 1, the region detectionblock Reg_Detect receives the segmented input image Img_(S) and carriesout a division phase that makes it possible to divide this image, pixelby pixel, into one or more regions forming part of a set of predefinedregions that differ from each other on the basis of the type of content,producing an output image Img_(Reg) divided into one or more regions.

Preferably, the predefined set of regions comprises regions of differenttypologies that differ from each other on account of the type of contentand are identified as: “Sky,” “Farthest mountain,” “Far mountain,” “Nearmountain,” “Land” and “Other.” Nevertheless, regions of different typescan also be taken into consideration.

It should be noted that the regions listed above are typically presentin panoramic images, landscapes or, in any case, outdoor photographs.

In a particularly preferred embodiment, the division into regions iseffected by establishing whether or not a pixel of the digital imagebelongs to a respective region on the basis of an operation analyzingthe parameters of that pixel. The analysis operation is carried out byverifying whether one or more parameters of the pixel satisfypredetermined conditions or logico-mathematical relationships or boththat characterize membership of the respective region. The conditionsand the logico-mathematical conditions that characterize membership of apredefined region are derived empirically from a statistical observationof images that depict real scenes.

Preferably, the analyzed parameters will correspond to digital valuesrepresentative of at least one of the following components of the pixel:red chromatic component, green chromatic component, blue chromaticcomponent, hue and intensity. Other parameters similar to those justcited may however be analyzed.

After the analysis operation, the division phase also includes anoperation of assigning to the analyzed pixel, i.e. associating it with adata item that identifies the respective region to which it belongs.

In an advantageous embodiment, the identification data item comprises adigital value, in practice a gray level.

For example, with pixels belonging to predefined regions that on thebasis of their content can be considered to be nearer to an observationpoint of the image there are associated respective darker gray levelsthan those of pixels belonging to more distant regions. Obviously, achoice of the opposite type is wholly equivalent.

It should be noted that the set of digital values determined by theoperation of assigning a digital value to the pixels of the digitalimage to be divided corresponds to a gray-level digital image dividedinto regions.

In a particular example, on the assumption that the pixels comprise8-bit digital values, the digital values (i.e. the gray levels) to beassigned to the pixels on the basis of the region to which they belongare, for example, as shown in the following table:

REGION VALUE Sky 0 Farthest mountain 50 Far mountain 80 Near mountain100 Land 200 Other 255

In a particularly preferred embodiment, the division into regions iseffected by analyzing the image pixel by pixel. For every pixel of thesegmented image there are extracted the values of the components R, G, Band the components H (Hue) and I (Intensity) and on the basis of thesevalues it is decided whether or not a pixel belongs to one of theregions listed above is then established on the basis of these values.

The characterization of the regions is effected by means of heuristicrules and, more particularly, by conditioning rules. The conditions arerepresented by intervals and logico-mathematical relationships betweenthe values R, G, B, H and I, while the corresponding actions arerepresented by the association with the analyzed pixels of a gray-levelvalue.

In a particularly preferred embodiment, the phase of dividing thesegmented image performed in the division block Reg_Detect comprises theoperations of:

application of a median filter, preferably 5×5, to the segmented imageImg_(S) for obtaining a filtered segmented image F_Img_(S);

extraction of the digital values associated with the components RGB andHSI of the filtered segmented image F_Img_(S);

association with each pixel of the filtered segmented image F_Img_(S) ofa gray level; associating a gray level to each pixel of the segmentedimage one obtains in practice a gray-level image temp_Img_(Reg);

application of a median filter, preferably 5×5, to the gray-level imagetemp_Img_(Reg) to obtain a final gray-level image Img_(Reg).

Advantageously, the use of median filters makes it possible to reducethe noise component present in both the input image and the output imageof the block for the division into regions.

FIG. 4 shows the flow chart of a preferred embodiment of the operationof assigning a gray-level I_(r)(x,y) to a pixel p(x,y) of the filteredsegmented image F_Img_(S) to obtain a corresponding pixel I_(r)(x,y) ofthe gray-level image temp_Img_(Reg).

A check is carried out in a first control phase Ck1 for establishingwhether or not the pixel p(x,y) belongs to a region of the type “Nearmountain” indicated in the figure by the reference “r1.”

If on the basis of this check the pixel p(x,y) is found to belong tothis region, a first phase of association Set1 assigns a gray levelhaving a digital value equal to 100 to the pixel I_(r)(x,y) of thegray-level image temp_Img_(Reg). In that case the association operationterminates and a new pixel is taken into consideration. Otherwise, asecond control phase Ck2 is carried out to establish whether or not thepixel p(x,y) belongs to a region of the type “Far mountain,” indicatedin the figure by the reference “r2.”

If on the basis of this check the pixel p(x,y) is found to belong tothis region, a second phase of association Set2 assigns a gray levelhaving a digital value equal to 80 to the pixel I_(r)(x,y) of thegray-level image temp_Img_(Reg). In that case the association operationterminates and a new pixel is taken into consideration.

Otherwise, a third control phase Ck3 is carried out for establishingwhether or not the pixel p(x,y) belongs to a region of the type “Farmountain,” indicated in the figure by the reference “r3.”

If on the basis of this check the pixel p(x,y) is found to belong tothis region, a third phase of association Set3 assigns a gray levelhaving a digital value equal to 50 to the pixel I_(r)(x,y) of thegray-level image temp_Img_(Reg). In that case the association operationterminates and a new pixel is taken into consideration.

In a preferred embodiment the conditions that make it possible toestablish whether or not the pixel p(x,y) belongs to a region of thetype “Near mountain” (“r1”) are expressed by the following Booleancondition:

B(x, y) ≥ G(x, y)  AND  I(x, y) ≤ (mean_I − 15)AND  B(x, y) ≤ (G(x, y) + 50)  AND  I(x, y) ≤ 90  ${{with}\mspace{14mu}{mean\_ I}} = {\left( {\sum\limits_{y = 0}^{9}{\sum\limits_{x = 0}^{W - 1}{C_{xy}{I\left( {x,y} \right)}}}} \right)/{\sum\limits_{y = 0}^{9}{\sum\limits_{x = 0}^{W - 1}C_{xy}}}}$$\left\{ \begin{matrix}{C_{xy} = 0} & {{{when}\mspace{14mu}{B\left( {x,y} \right)}} < {G\left( {x,y} \right)}} \\{C_{xy} = 1} & {{{when}\mspace{14mu}{B\left( {x,y} \right)}} \geq {{G\left( {x,y} \right)}.}}\end{matrix} \right.$

Likewise in a preferred embodiment, the conditions that make it possibleto establish whether or not the pixel p(x,y) belongs to a region of thetype “Far mountain” (“r2”) are expressed by the following Booleancondition:B(x,y)≧G(x,y) AND I(x,y)≦(mean_(—) I−15) AND B(x,y)≦(G(x,y)+50) ANDI(x,y)≦130.

While the conditions for establishing whether or not the pixel p(x,y)belongs to a region of the type “Farthest mountain” (“r3”) are expressedby the following Boolean condition:B(x,y)≧G(x,y) AND I(x,y)≦(mean_(—) I−15) AND B(x,y)≦(G(x,y)+50) ANDI(x,y)>130.

In general, the condition that is common to the greater part of thepixels that represent mountains is the one of having the component Bgreater than the component G, but limited by G+50.

The quantity mean_I constitutes an estimate of the mean value of theluminous intensity of the pixels that should belong to a region of thetype “Sky.” It is calculated as the mean of the luminous intensity ofthe pixels of the first ten rows of the image having the component Bgreater than the component G. The check of the value of the intensity Iwith respect to mean_I−15 makes it possible to avoid pixels belonging toa region of the type “Sky” being mistaken for a pixel of a region of thetype “Mountain.”

In particular, the differentiation between the three types of “Mountain”is effected by exploiting the concept of atmospheric perspectiveintroduced for the first time by Leonardo in the famous painting of theMona Lisa. In fact, he had noted that the most distant mountains, giventhat they had the same chromatic hue as their nearest counterparts,appeared to be clearer (and therefore had a greater luminous intensity).This phenomenon can be explained by considering that the quantity ofatmospheric dust that becomes interposed between an observer and anobject increases with the distance of the object from the observer.

In this manner the more distant objects have colors nearer to white andtherefore have a luminous intensity value greater than the nearerobjects.

The “Farthest mountain” region is therefore characterized by a luminousintensity value I greater than 130, “Far mountain” has a luminousintensity value I comprised between 90 and 130, while “Near mountain”representing the nearest mountain is characterized by a luminousintensity value I of less than 90.

Coming back to the flow chart of FIG. 4, if on the basis of the checkscarried out in the phases Ck1, Ck2 and Ck3 the pixel p(x,y) is found notto belong to one of the regions of the “Mountain” type, a fourth controlphase Ck4 is undertaken to ascertain whether or not the pixel p(x,y)belongs to a region of the “Land” type indicated in the figure by thereference “r4.”

If on the basis of this check the pixel p(x,y) is found to belong tothis region, a fourth phase of association Set4 assigns a gray levelhaving a digital value equal to 200 to the pixel I_(r)(x,y) of thegray-level image temp_Img_(Reg). In that case the association operationterminates and a new pixel is taken into consideration.

In a preferred embodiment the conditions for checking whether or not thepixel p(x,y) belongs to a region of the “Land” type (“r4”) are expressedby the following Boolean conditions:|R(x,y)−G(x,y)|≦10 AND R(x,y)≧(B(x,y)+55)|R(x,y)−G(x,y)|≦30 AND |R(x,y)−B(x,y)|≦30 AND 40≦I(x,y)≦80R(x,y)≧(B(x,y)+45) AND R(x,y)≧G(x,y)R(x,y)≧G(x,y) AND G(x,y)≧B(x,y) AND 25≦H(x,y)≦50G(x,y)≧(R(x,y)+10)G(x,y)≧(B(x,y)+15) AND I(x,y)≧75|R(x,y)−G(x,y)|≦12 AND G(x,y)>B(x,y) AND R(x,y)>B(x,y).

The pixel is deemed to belong to a region of the “Land” type if one ofthe aforesaid conditions is found to be satisfied.

These conditions describe the family of lands that can be present in animage of the “landscape type.” The first condition describes the colorstypical of a beach. The next three conditions represent colors of landswith chromatic gradations tending towards dark brown and red. Thepenultimate condition describes the colors typical of green meadowscharacterized by having the component G greater than the others and arather high luminous intensity (greater than 75). The last condition hasbeen inserted to represent the cultivated fields (wheat fields, forexample).

If none of the aforementioned six conditions is satisfied, a fifthcontrol phase Ck5 is carried to ascertain whether or not the pixelp(x,y) belongs to a region of the “Sky” type indicated in the figure bythe reference “r5.”

If on the basis of this check the pixel p(x,y) is found to belong tothis region r5, a fifth phase of association Set5 assigns a gray levelhaving a digital value equal to 0 to the pixel I_(r)(x,y) of thegray-level image temp_Img_(Reg). In that case the association operationterminates and a new pixel is taken into consideration.

In a preferred embodiment the conditions for checking whether or not thepixel p(x,y) belongs to a region of the “Sky” type (“r5”) are expressedby the following Boolean conditions:B(x,y)≧(G(x,y)+30) AND G(x,y)≧R(x,y)B(x,y)≧G(x,y) AND G(x,y)≧R(x,y)B(x,y)≧R(x,y) AND R(x,y)≧G(x,y)I(x,y)≧200 AND |B(x,y)−G(x,y)|≦30 AND |R(x,y)−G(x,y)|≦30

The pixel is deemed to belong to a region of the “Sky” type if one ofthe aforesaid four conditions is found to be satisfied.

The first three conditions represent three variations of blue skies inwhich the predominant component is B(x,y). They describe the most commontypes of sky. The last condition represents a range of cloudy skiescharacterized by a gray color. Given the nature of the chromaticgradations of gray, there is no need for the presence of a componentgreater than the others. Furthermore, there is present a check of thevalue I to avoid that pixels that really belong to one of the regions ofthe “Mountain” type are labeled as “Sky.”

The choice of the threshold value equal to 200 derives from theobservation of the value I of cloudy skies present in the images. Infact, they are characterized by being particularly luminous. If none ofthe aforesaid four conditions is satisfied, the pixel p(x,y) isautomatically assigned to a region of the “Other” type and a sixth phaseof association Set6 assigns a gray level having a digital value equal to255 to the pixel I_(r)(x,y) of the gray-level image temp_Img_(Reg); theassociation operation then terminates and a new pixel is taken intoconsideration.

As can be noted, the conditions for belonging to the regions can bebriefly summarized as follows:

conditions whether or not the parameters of the pixel belong to a giveninterval, or

mathematical relationships between parameters of one and the same pixel,or

any combination of the aforementioned conditions and combinations.

In FIGS. 5A through 5C the reference Img_(RGB) indicates an output imageof the color reconstruction block Col_Rec, while Img_(S) and Img_(Reg)indicate the corresponding output images of, respectively, thesegmentation block Img_Seg and the block Reg_Detect that divides theimage into regions. As may be noted, the image Img_(Reg) is divided intofour regions, respectively, from top to bottom: of the “Sky” type (r5),the “Farthest mountain type (r3), the “Near mountain” type (r1) and the“Land” type (r4).

Coming back to the block diagram of FIG. 1, a class decision block makesit possible to associate with the image Img_(Reg) a class I_Classforming part of a predefined set of image classes CL₁, CL₂, . . . ,CL_(n). In a particularly preferred embodiment the set of predefinedclasses includes the following classes: CL₁=“outdoor with geometricappearance,” CL₂=“outdoor without geometric appearance (or panoramas),”CL₃=“indoor” and similar.

It has been found that each class may be characterized by means of anappropriate simplified model obtained on the basis of a statisticalobservation of some characteristics typically present in imagesbelonging to the class.

In a particularly preferred embodiment, a predefined Class CL₁ ischaracterized by a respective model comprising one or more predefinedsequences (i.e. successions in accordance with a particular disposition)of regions that can be typically observed along a predetermined scanningdirection in images belonging to this class. In particular, eachpredefined sequence includes a string of identifying labels (i.e. data)assigned to regions typically found along the predetermined scanningdirection.

For example, considering the following table of labels identifyingcorrespondences between regions and gray levels:

REGION LABEL GRAY LEVEL R5 = Sky s 0 R3 = Farthest mountain M 50 R2 =Distant mountain M 80 R1 = Nearby mountain M 100 R4 = Land L 200 R6 =Other X 255it was found that the sequences of regions that could be observed with acertain statistical frequency in images belonging to the classCL₁=“outdoor without geometric appearance (or panoramas)” are expressedby the following strings of identifying labels:

“S”

“sm”

“sl”

“sml”

“m”

“ml”

“l”

“sms”

“smsl”

“msl”

“mls”

“ls”

The label “s” placed after the labels “m” and/or “l” indicatessituations in which a generic water surface is present in the lower partof the image.

It has also been found that the images belonging to the predefined classCL₂=“outdoor with geometric appearance” present region sequences that donot belong to the set of sequences listed above, but neverthelesscomprise a region of type s, i.e. “Sky,” in the upper part.

Lastly, the remaining regions are characteristic of images belonging tothe class CL₃=“indoor.”

Advantageously, in the class decision block, which is shown in greaterdetail in FIG. 6, there is carried out a phase f₁, for acquiringinformation from the image Img_(Reg) divided into regions to obtaininformation about the regions present in it.

Preferably, the information acquired in this phase f₁ concerns: thenumber of transitions between regions and/or the type of regions presentand/or the respective order of disposition in accordance with apredetermined scanning direction.

In an even more preferred embodiment, the information acquisition phasef₁ comprises an operation S_Scan of acquiring one or more sequencesS_(j) of regions from the image Img_(Reg) divided into regions. Thisoperation comprises, for example, one or more scannings of the imagedivided into regions along one or more predefined directions. In greaterdetail, during the scanning along a particular direction the identifyinglabels assigned to the regions encountered during the scanning arememorized in a string S_(j) of characters.

Preferably, the scanning of the image is a scanning of the verticaltype, from top to bottom for example, of the pixels belonging to one ormore columns of the image. For example, the columns are selected atregular intervals starting from the top left-hand corner of the image.Hereinafter the present description will make reference to the scanningalong the columns of the image without any limitation.

Preferably, the operation of acquiring the number N of acquired regions,i.e. the number N of columns subjected to scanning, is comprised between5 and 10. In fact, it has been found that even though the acquisition ofa large number N of sequences S_(j) provides a larger quantity ofinformation about the image that is to be classified, it does not alwaysincrease the probability of having a correct classification. In manycases, in fact, the experiments that envisaged the acquisition of alarge number of sequences S_(j) provided the same results as experimentscarried out with a smaller number of acquired sequences.

Preferably, the gray levels associated with the pixels and theidentifying labels encountered in the scanning direction are memorizedduring the scanning. For example, an identifying label is memorized in astring S_(j) whenever it differs from the last identifying labelmemorized in the string S_(j). Obviously, the label associated with thefirst region encountered is always memorized. Even more preferably, alabel is memorized in the string S_(j) whenever the number of pixelsencountered during the scanning of a column exceeds a predeterminedthreshold value H_(min) that corresponds to a minimum number of pixelschosen in manner proportional to the height H of the image. Preferably,the threshold value H_(min) is substantially comprised between 5% and15% of the vertical dimension H of the image. In a preferred embodiment,the information acquisition phase also comprises a counting operationJ_Count along one or more predetermined directions of the number C_(j)of transitions, i.e. passages (or jumps), between regions of differenttypes of the image divided into regions.

For example, the counting phase is carried out directly during thescanning of a sequence S_(j) of the image divided into regions byincreasing a counter C_(j) every time a transition between differentregions is detected. In this case a corresponding counter C_(j) can beassociated with every string S_(j), thus obtaining a data pair (S_(j),C_(j)) for every scanning of the image.

For example, a transition is detected during the scanning along a columnof the image divided into regions whenever there is read a value (graylevel) of a pixel different from the last previously read value.

It should be noted that, given a highly fragmented image divided intoregions, even though the number of labels in the acquired strings S_(j)may be limited on account of the control of the minimum number of pixels(H_(min)), the value of the corresponding transition counter C_(j) maybe high.

FIG. 7 shows the image divided into regions of FIG. 5 together with fivecolumns c1, c2, c3 c4, c5 selected for scanning.

On the assumption that the columns c1, c2, c3 c4, c5 are scanned fromtop to bottom, the label strings S_(j) read and the number and thetransitions counted by the corresponding counters C are shown column bycolumn in the following table:

Column String Number of transitions Column c1 S₁ = “sml” C₁ = 3 Columnc2 S₂ = “sml” C₂ = 3 Column c3 S₃ = “sml” C₃ = 3 Column c4 S₄ = “ml” C₄= 3 Column c5 S₅ = “ml” C₅ = 2

Coming back to the block diagram of FIG. 6, in the class decision blockthere is also carried out an analysis phase f₂ of the informationacquired in the acquisition phase f₁. The association of one of thepredefined classes CL₁, CL₂, CL₃ with the image divided into regions isobtained on the basis of the result of this analysis phase f₂.

In particular, the analysis phase f₂ includes a phase Cmp of comparingthe information (S_(j), C_(j)) obtained in the acquisition phase1 withthe models that characterize the predefined classes.

The preferred embodiment in which information about the region sequencesS_(j) and the transition counters C_(j) is acquired during theinformation detection phase f₁ also includes an operation S_Discard forselecting a subset of the N acquired sequences S_(j) comprisingsequences S_(j) associated with a respective transition counter C_(j)containing a value smaller than or, at the most, equal to apredetermined limit value C_(th). Furthermore, the non-selectedsequences are preferably discarded. For example, it has been found inthe course of experimental tests that good results are obtained with alimit value C_(th) approximately equal to ten.

The comparison operation Cmp is such as to verify whether the selectedacquired sequences, i.e. the non-discarded sequences, are present amongthe predefined sequences of a model characterizing a predefined class.

Preferably, a predefined class CL₁, CL₂, CL₃ is associated with theimage whenever among the predefined sequences of the model thatcharacterizes the predefined class there is present a number of selectedsequences greater than a predetermined percentage of the total number Nof acquired sequences S_(j).

In the example of FIG. 6 the comparison operation Cmp includes a firstcontrol operation Cmp1 in which there is determined the number N of theacquired and not discarded sequence S_(j) that are comprised in the setof predefined sequences belonging to the model of the Class CL₁=“outdoorwithout geometric appearance (or panoramas).” Whenever the determinednumber N₁ is greater than or equal to a predetermined percentage of thetotal number N of acquired sequences S_(j), a first associationoperation Set CL₁ assigns the predefined class CL₁ to the image. Inpseudo code, whenever:N ₁ ≧R ₁ ×NthenI_Class=CL₁wherein R₁ is a number comprised between 0 and 1. The value of R₁ playsa fundamental part in associating the predefined class CL₁, because itfurnishes a measure of the precision level of the entire method. It isobvious to note that low values of R₁ (smaller than 0.5, for example)make it possible to classify an image as belonging to the predefinedclass CL₁ with greater “ease.” In these cases one therefore risks“taking for good” even images that in reality are not so.

On the other hand, utilizing a high value of R1 (greater than 0.8, forexample) possibly risks an excessive thinning of the tolerance marginthat has to be assured for a heuristic method.

From the acquired experience and the analyzed results it was found thatvalues of R₁ substantially comprised between 0.6 and 0.8 make itpossible to obtain sufficiently accurate and trustworthy results.

Whenever the number N₁ does not satisfy the above condition, a secondcontrol operation Cmp2 is carried out to determine the Number N₂ of theacquired and non-discarded sequences S_(j) that are comprised in the setof predefined sequences belonging to the model of the class CL₂=“outdoorwith geometric appearance.” When the determined number N₂ is greaterthan or equal to a predetermined percentage of the total number Nacquired sequences S_(j), a second association operation assigns thepredefined Class CL₂ to the image. In pseudo code, when:N ₂ ≧R ₂ ×NthenI_Class=CL₂wherein R₂ is a number comprised between 0 and 1.

Since in this case the predefined class includes sequences of regionsthat begin with the region “Sky,” the tolerance margin determined by theparameter R₂ is very ample, and experimental results have shown thatvalues of R2 substantially comprised between 0.1 and 0.5 make itpossible to obtain sufficiently accurate and trustworthy results. In aparticularly preferred embodiment, R2 is substantially equal to 02.

When the number N₂ does not satisfy the above condition, a thirdassociation operation Set_CL3 is carried out to assign to the image thepredefined class Cl₃=“indoor,” which therefore contains all thesequences of regions not comprised in the models that characterize thepredefined classes CL₁ and CL₂.

The proposed method has such advantages as automaticity, computationalsimplicity and efficiency.

Automaticity is a very important property of any image processingprocedure. Indeed, the construction of a technique capable of providingreliable results by eliminating interaction with the human userrepresents a step forward in the global process of automation of thepresently available technologies. Furthermore, it implies thepossibility of aggregating image classification with other automaticprocesses for the purpose of creating advanced multi-process systems.

Simplicity and efficiency are two characteristics that do not always gohand in hand. In the method in question the problem of imageclassification is resolved by having recourse to a multi-stage systemmade up of a series of functional blocks that are relatively simplebecause they are implemented by means of algorithms based on ratherelementary heuristic characterizations and statistical properties.Efficiency depends primarily on having used input images in CFA formatand having made provision for color reconstruction with sub-sampling,thereby making it possible to work with images having dimensions equalto half the original ones, and having used the least number of possibleoperations (see, for example, the simplicity of the class decisionblock).

The method in accordance with the present invention may be directlyutilized, for example, in such portable or fixed image acquisitiondevices as: digital photo/video cameras and the like, multimediacommunication terminals capable of acquiring and/or receiving digitalimages. Furthermore, the method in accordance with the present inventioncan be executed as a program or a program portion in a processingdevice, a personal computer for example. This program can be memorizedin any appropriate manner, for example in a memory or on such amemorization support as a floppy disk or a CD ROM.

All of the above U.S. patents, U.S. patent application publications,U.S. patent applications, foreign patents, foreign patent applicationsand non-patent publications referred to in this specification and/orlisted in the Application Data Sheet, are incorporated herein byreference, in their entirety.

Obviously, with a view to satisfying contingent and specific needs, aperson skilled in the art will be able to introduce numerousmodifications and variants into the classification method describedabove, though all contained within the protection limits as defined bythe claims attached hereto and the equivalents thereof.

1. A method for associating with a digital image a class of a pluralityof classes characterized by respective models, the method comprising thefollowing phases in sequence: analyzing a digital image pixel by pixelto determine parameters of each pixel by verifying that each parametersatisfies at least one of pre-established conditions,logico-mathematical relationships, or both pre-established conditionsand logico-mathematical relationships that are associated withrespective predefined regions comprising sky, land, and mountain, anddividing the digital image pixel by pixel into a respective regionbelonging to a set of regions comprising sky, far mountain, nearmountain, and land that are predefined and different from each other onthe basis of their type of content based on the analysis, includingassigning to each pixel a data item identifying the respective region towhich the pixel belongs, said identifying data item comprising a digitalvalue, that comprises a gray level, a set of digital values assigned tothe pixels of the digital image in said assignment operationcorresponding to a digital gray-level image divided into regions,acquiring from the digital image divided into regions informationrelating to the regions to be found in it, comparing the acquiredinformation with at least one model characterizing a respective class ofsaid plurality of classes, associating with the digital image a class ofsaid plurality of classes on the basis of said comparison phase, and aphase of segmenting the digital image prior to the dividing phase, thedigital output image of the segmentation phase divided into one or moreareas that are chromatically substantially homogeneous and comprising areduced range of colors as compared with the input image of thesegmentation phase.
 2. The method of claim 1, including prior to thedivision phase a phase of chromatic reconstruction that, starting froman input image in CFA format, provides an output image.
 3. The method ofclaim 2, wherein the chromatic reconstruction phase comprises anoperation of sub-sampling the digital image in CFA format in such amanner as to obtain a digital output image having a smaller resolutionthan the input image.
 4. The method of claim 1, wherein the segmentationphase is substantially carried out in accordance with a “mean shiftalgorithm” technique.
 5. The method of claim 1, wherein the segmentationphase employs a low segmentation resolution, the digital output image ofthe segmentation phase comprising a significantly reduced range ofcolors as compared with its digital input image and includingpredominant colors present in the input image.
 6. The method of claim 1,wherein said pre-established conditions and logico-mathematicalrelationships are empirically derived from a statistical observation ofimages of real scenes.
 7. The method of claim 1, wherein saidpre-established conditions and logico-mathematical relationshipscomprise conditions for ascertaining whether parameters belong tointervals and comprise mathematical relationships between parameters. 8.The method of claim 1, wherein said parameters correspond to digitalvalues representative of at least one of the following pixel components:red chromatic component, green chromatic component, blue chromaticcomponent, hue, and intensity.
 9. The method of claim 1, wherein saidset of predefined regions includes “Farthest mountain.”
 10. The methodof claim 1, wherein pixels belonging to regions that on the basis oftheir content are closer to an image observation point are associatedrespective gray levels that are darker than those associated with pixelsbelonging to more distant regions.
 11. The method of claim 1, whereinthe phase of dividing into regions comprises the following operations:application of a median filter to the digital image to obtain a filtereddigital image; extraction of the digital values associated with RGB andHSI components of the filtered digital image; assignment to each pixelof the filtered digital image of a respective gray level for obtaining aprovisional gray-level image divided into regions; application of amedian filter to said provisional gray-level image for obtaining a finalimage divided into gray-level regions.
 12. The method of claim 1,wherein said set of predefined classes comprises at least one of thefollowing classes: “outdoor with geometric appearance,” “outdoor withoutgeometric appearance,” and “Indoor.”
 13. The method of claim 1, whereineach predefined class is characterized by means of a suitable simplifiedmodel obtained on the basis of a statistical observation of somecharacteristics typically present in images belonging to the class. 14.The method of claim 1, wherein at least one predefined class ischaracterized by a respective model comprising one or more predefinedsequences of regions typically detected along a predetermined scanningdirection in images forming part of said predefined class.
 15. Themethod of claim 14, wherein a predefined sequence includes a string ofidentifying labels assigned to regions typically detected along thepredetermined scanning direction.
 16. The method of claim 1, wherein theinformation acquired in the acquisition phase regards the number ofregions or their type or the respective order in which they are arrangedalong a predetermined scanning direction.
 17. The method of claim 1,wherein the information acquisition phase comprises an operation ofacquiring one or more sequences of regions from the image divided intoregions by means of an operation of scanning the image divided intoregions along at least one predetermined scanning direction.
 18. Themethod of claim 17, wherein a string of identifying label charactersassigned to regions encountered during the scanning is memorized duringsaid operation of scanning along the predetermined scanning direction.19. The method of claim 17, wherein said scanning is a scanning of thevertical type of pixels belonging to one or more columns of the imagesdivided into regions.
 20. The method of claim 1, wherein saidinformation acquisition phase includes an operation of counting thenumber of transitions between different regions along at least onepredetermined direction of the image divided into regions.
 21. Themethod of claim 14, wherein said comparison operation verifies whetherone or more predefined sequences are present among the predefinedsequences comprised in a model characterizing said predefined class. 22.The method of claim 21, wherein said predefined class is associated withthe digital image when among the predefined sequences of the modelcharacterizing said predefined class there is present a number ofacquired sequences greater than a predetermined percentage of the totalnumber of said acquired sequences.
 23. A digital image acquisitiondevice inclusive of a classification block for associating a class of aplurality of predefined classes with an acquired digital image,characterized in that said classification block operates on the basis ofa method in accordance with claim
 1. 24. A non-transitory computerreadable medium whose contents contain a program or part of a computerprogram stored thereon that when executed configure the computer toassociate with a digital image a class of a predefined plurality ofclasses in accordance with the following method: analyzing a digitalimage pixel by pixel to determine parameters of each pixel by verifyingthat each parameter satisfies at least one of pre-establishedconditions, logico-mathematical relationships, or both pre-establishedconditions and logico-mathematical relationships that are associatedwith respective predefined regions comprising sky, land, and mountain,and dividing the digital image pixel by pixel into a respective regionbelonging to a set of regions comprising sky, far mountain, nearmountain, and land that are predefined and different from each other onthe basis of their type of content based on the analysis, includingassigning to each pixel a data item identifying the respective region towhich the pixel belongs, said identifying data item comprising a digitalvalue, that comprises a gray level, a set of digital values assigned tothe pixels of the digital image in said assignment operationcorresponding to a digital gray-level image divided into regions,acquiring from the digital image divided into regions informationrelating to the regions to be found in it, comparing the acquiredinformation with at least one model characterizing a respective class ofsaid plurality of classes, associating with the digital image a class ofsaid plurality of classes on the basis of said comparison phase, and aphase of segmenting the digital image prior to the dividing phase, thedigital output image of the segmentation phase divided into one or moreareas that are chromatically substantially homogeneous and comprising areduced range of colors as compared with the input image of thesegmentation phase.
 25. A method for classifying a digital image in aclass from a plurality of classes, each class having a respective modelassociated therewith, the method comprising: analyzing a digital imagepixel by pixel to determine parameters of each pixel by verifying thateach parameter satisfies at least one of pre-established conditions,logico-mathematical relationships, or both pre-established conditionsand logico-mathematical relationships that are associated withrespective predefined regions comprising sky, land, and mountain, anddividing the pixels into a respective region based on the analysis,including assigning to each pixel a data item identifying the respectiveregion to which the pixel belongs, said identifying data item comprisinga digital value, that comprises a gray level, a set of digital valuesassigned to the pixels of the digital image in said assignment operationcorresponding to a digital gray-level image divided into regions;acquiring from the digital image information relating to the regionsfound in the digital image; comparing the acquired information with atleast one model; and associating with the digital image a class on thebasis of the comparison.
 26. The method claim 25, comprising initially:receiving an input image in CFA format and performing a chromaticreconstruction on the input image to provide an output image; andsegmenting the output image by dividing the output image into one ormore areas that are chromatically substantially homogeneous and thatcomprise a reduced range of colors as compared with the output image toform an output digital image.
 27. The method of claim 26, wherein thechromatic reconstruction comprises subsampling the input image in CFAformat in such a manner as to obtain a digital output image having asmaller resolution than the input image.
 28. The method of claim 27,wherein the segmentation is substantially carried out in accordance witha mean shift algorithm technique that employs a low segmentationresolution in which a significantly reduced range of colors are presentas compared with the input image and including predominant colorspresent in the input image.
 29. A method for associating with a digitalimage a class of a plurality of classes characterized by respectivemodels, the method comprising the following phases in sequence: dividingthe digital image pixel by pixel into a respective region belonging to aset of regions comprising sky, mountain, and land that are predefinedand different from each other on the basis of their type of content,said division being effected by establishing whether or not a pixel ofsaid image belongs to a respective region on the basis of an operationof analyzing the parameters of this pixel, the analysis operation beingcarried out by verifying that said parameters satisfy pre-establishedconditions or logico-mathematical relationships or both that belong tothe respective region, including assigning to each pixel a data itemidentifying the respective region to which the pixel belongs, saididentifying data item comprising a digital value, that comprises a graylevel, a set of digital values assigned to the pixels of the digitalimage in said assignment operation corresponding to a digital gray-levelimage divided into regions, acquiring from the digital image dividedinto regions information relating to the regions to be found in it,comparing the acquired information with at least one modelcharacterizing a respective class of said plurality of classes,associating with the digital image a class of said plurality of classeson the basis of said comparison phase, and at least one predefined classis characterized by a respective model comprising one or more predefinedsequences of regions detected along a predetermined scanning directionin images forming part of said predefined class.
 30. The method of claim29, wherein a predefined sequence includes a string of identifyinglabels assigned to regions typically detected along the predeterminedscanning direction.